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About the Study 


This issue brief is part of a five-part series that reports findings 
from research conducted by ACT in collaboration with Region 
One Education Service Center, a provider of educational 
services to over 430,000 students in South Texas who are 
primarily Latinx and from low-income communities. The 
research was funded by a NewSchools Venture Fund grant 
awarded to ACT’s Center for Social, Emotional, and Academic 
Learning and ACT’s Center for Equity in Learning. The 
overarching goal of the research was to gain a better 
understanding of factors related to social and emotional 
learning (SEL), including program efficacy, implementation, 
and family engagement. This brief focuses on the efficacy of 
SEL lessons implemented during the 2019-2020 school year 
and examines their relationship to school-reported student 
outcomes such as grades, attendance, and conduct. The goals 
of the study were to better understand the effects of SEL 
programming on student outcomes and to explore other factors 
related to efficacy, such as lesson dosage and fidelity of 
implementation. 


So What? 


The current study investigated how SEL lessons relate to 
social and emotional and academic outcomes in a group of 
students from primarily Latinx, low-income communities with a 
substantial percentage of English-language learners. Despite 
disruption due to the COVID-19 pandemic, the fidelity of 
implementation was relatively high, but the dosage was 
somewhat lower. Although results were mixed, hours spent on 
the SEL lessons helped to predict two of the target outcomes: 
improved math grades and decreased odds of a disciplinary 
incident in 2020. 


Now What? 


In contrast to the positive reactions of students to the SEL 
lessons, results from the current study examining more distal 
outcomes were mixed. These mixed results may be due to the 
relatively small amount of SEL programming students received 
and the relatively short delay between the lessons and 
measurement of the outcomes. These results suggest time and 
effort are required to provide the sustained implementation of 
programming that may be necessary to produce long-term 
benefits to students. Replicating these results with a more 
traditional instructional approach and increasing the amount of 
SEL programming are important next steps for future research. 
Findings from the current study also underscore the 
importance of a structured implementation plan and continued 
monitoring of the fidelity and dosage of implementation. 


ACT 


ACT.org/research 
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Relationships Between Social and Emotional Skill Building 


Lessons and Student Outcomes 


Social and emotional (SE) skills are defined as interpersonal, self-regulatory, and task-related 
behaviors that are important for adaption to and successful performance in educational and 
workplace settings (Casillas, Way, & Burrus, 2015). These skills are related to important 
outcomes, including improved attendance, positive behaviors, improvements in school climate, 
and academic achievement (e.g., Claro & Loeb, 2019; Domitrovich, Durlak, Staley, & 
Weissberg, 2017). Further, a growing body of research shows that these skills can improve in 
response to social and emotional learning (SEL) interventions (Mahoney, Durlak, & Weissberg, 


2018). 


Given the importance of supporting these skills, the students in this study received 
supplemental SEL lessons following the Mosaic™ by ACT®: Social Emotional Learning 
assessment. After taking the assessment, students are given reports with their results and links 
to lessons that target five SE skills: Sustaining Effort, Getting Along With Others, Maintaining 
Composure, Keeping an Open Mind, and Social Connection. These lessons can be used in 
universal classroom settings and are designed to support students’ understanding of and ability 
to demonstrate SE skills. Skill definitions and examples of lessons for these skills are described 


in Table 1. 


Table 1. Description of Each Skill and Examples of Required and Optional Lessons for Each of 


the Skills 


How a student’s 


actions 
demonstrate: 


Description of a 


required introductory 
lesson 


Description of an 
F-Yeleliaveyar-1 im (st-s-xe) a) 


Description of the required 
final lesson 


Sustaining 
Effort 


Getting Along 
With Others 


Maintaining 
Composure 


Diligence, effort, 
organization, self- 
control, and 
compliance with 
the rules. 


Collaboration, 
empathy, 
helpfulness, trust, 
and 
trustworthiness. 


Stress 
management, 
emotional 
regulation, a 
positive response 
to setbacks, and 
poise. 


The introductory lesson 
and discussion aim to 
introduce students to 
Sustaining Effort and why 
it is important. 


In the introductory group 
activity, students identify 
individuals who 
demonstrate Getting 
Along With Others and 
understand how 
exemplifying this skill has 
helped them succeed. 


In one introductory 
lesson, students learn 
about the key differences 
between Maintaining 
Composure and 
Sustaining Effort. 


One of the additional 
lessons contains stories 
about key historical figures 
who succeeded because of 
their persistence. 


In one of the additional 
lessons, students learn and 
practice how to ask the right 
questions that will enable 
them to understand others’ 
points of view. 


In one of the additional 
lessons, students learn how 
to identify and manage their 
emotions and the factors 
that cause them. 


The final lesson allows 
students to demonstrate 
their knowledge of 
Sustaining Effort and apply it 
to their own lives. 


In the final lesson, students 
learn how to effectively help 
others resolve conflicts by 
using all of the skills they 
have used throughout the 
unit. There is also a take- 
home essay component to 
the lesson. 


In the final lesson, students 
can demonstrate their 
knowledge of Maintaining 
Composure by identifying a 
Maintaining Composure role 
model and sharing how this 
role model exemplifies 
Maintaining Composure. 
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For each skill, three types of lessons were provided: introductory, additional, and culminating. A 
mandatory introductory lesson defined each skill and provided some basic information, the 
additional lessons provided more detail about each skill and involved activities to improve the 
skill, and the mandatory culminating lessons wrapped up the lessons for each skill by using the 
skills students developed during the prior lessons. 


All students can benefit from universal SEL programming. However, it may be of increased 
importance among students from underserved backgrounds, such as students from low-income 
communities, students of color, and students who would be the first generation in their families 
to attend college. These students need additional supports because they often face significant 
barriers to college entry and degree attainment (Zuo, Mulfinger, Oswald, & Casillas, 2018). 
Students from underserved backgrounds may experience lower academic expectations, lower 
college readiness rates due to unfinished learning, and higher levels of educator stress (ACT, 
2016; McKown & Weinstein, 2008; Peng & Lee, 1994). Additionally, students of color often face 
exclusionary discipline practices that may lead to withdrawal from learning in the classroom and 
eventual dropout or failure to graduate on time. (Gilliam, Maupin, Reyes, Accavitti, & Shic, 2016; 
Gregory, Skiba, & Noguera, 2010; Morris & Perry, 2016). SEL can help to improve teachers’ SE 
skills, increase students’ sense of belonging in academic settings, and ultimately contribute to 
creating more equitable learning environments (Jagers, Rivas-Drake, & Williams, 2019). 
Therefore, the current paper reports relationships between SEL lessons and target outcomes 
within a group of primarily Latinx learners from low-income communities. 


To examine the efficacy of the SEL lessons used in this study, we use ACT’s efficacy framework 
(Mattern, 2019). This framework describes several sources of efficacy data aligned to 
Kirkpatrick’s (1976) evaluation model. This model organizes possible outcomes into four levels. 


Level I: The extent to which learners find the lessons engaging and relevant 


Level Il: The extent to which learners acquire the intended knowledge and skills from the 
lessons 


Level III: The extent to which learners apply content from the lessons 
Level IV: The extent to which target outcomes are improved based on the lessons 


Brief 2 (Student Reactions and Perceived Gains to Social and Emotional Learning Lessons) in 
this series reports initial evidence of the efficacy of these SEL lessons from a subset of students 
within this larger study. Students in this study were surveyed following SEL lessons on 
Sustaining Effort and either Getting Along With Others or Maintaining Composure. In response 
to survey items measuring outcomes at Kirkpatrick’s Levels | through III, students reacted 
positively to the lessons, demonstrated evidence of improved content knowledge on various 
measures, and reported confidence in applying content from the lessons both inside and outside 
of school. In the current report, we expand on these promising results by examining how these 
lessons relate to student outcomes consistent with Level IV. 


To examine how these lessons impact other target outcomes, we chose a subset of schools 
from Region One to administer the SEL lessons. The implementation of these SEL lessons 
aimed to follow previously proposed recommendations for interventions aiming to support skill 
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building (Durlak, Weissberg, & Pachan, 2010). Specifically, SEL implementation should follow 
SAFE (sequenced, active, focused, explicit) recommendations. For more information about 
these recommendations and their implementation in the current study, see Table 2. 


Table 2. Definition of SAFE Recommendations and Implementation in the Current Study 


InX-Xere)anlant=yareretice) a} 


Sequenced 


Active 


Focused 


Explicit 


DY=Vitaliateyal 


Incorporating a 
specified order of 
activities generally via 
lessons or program 
materials. 


Giving students a 
chance to act on the 
material; for instance, 
by practicing behaviors 
and receiving 
feedback. 


Having dedicated time 
for specific skill 
development. 


Specifying learning 
objectives with specific 
target skills. 


Example of implementation in the current study 


Lessons on each skill follow a specified order, 
starting with introductory lessons, continuing with 
four additional lessons chosen from the list, and 
ending with a culminating activity. See Figure 1 for 
more detail. 


Lessons for each of the skills incorporate active 
learning. For instance, one lesson on Sustaining 
Effort involves students practicing setting their own 
academic and extracurricular goals. 


Each of the lessons is focused on supporting the 
development of a specific skill, providing dedicated 
time for each skill. 


Specific student learning objectives accompany 
each lesson. For instance, the lesson on sustaining 
effort described above corresponds to the goal 
“students will practice setting their own academic 
and extracurricular goals.” 


Previous research shows that using these four recommendations moderated the effects of 
programming on several outcomes. These include increased positive school behaviors such as 
leadership, achievement test scores, and grades, as well as decreased behaviors disruptive to 
learning, such as disciplinary referrals (Durlak et al., 2010). Additionally, a meta-analysis 
examining the effects of school-based SEL programs found that when all four SAFE 
recommendations were followed, school-based SEL programs led to significantly improved 
skills, attitudes, social behavior, and academic performance, as well as decreased conduct 
problems and lowered emotional distress. In contrast, school-based SEL programs that failed to 
follow at least one of the recommendations only significantly improved attitudes, increased 
academic performance, and decreased conduct problems (Durlak, Weissberg, Dymnicki, Taylor, 


& Schellinger, 2011). 


Although a systematic implementation plan is necessary, 
monitoring how that plan is carried out in a specific study is 
essential for interpreting the program’s impact. Two important 
aspects of implementation are fidelity and dosage (Durlak & 
DuPre, 2008). Fidelity refers to the extent to which the 
program is delivered as intended in the study. Dosage refers 
to how much of the intended program is delivered to the 
participant. Problems with these and other aspects of 


Fidelity: The extent to which 
the program is delivered as 
intended in the study 


Dosage: How much of the 
intended program is delivered 
to the participant 
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implementation are associated with less effective program outcomes (Durlak & DuPre, 2008; 
Durlak et al., 2011). Because implementation can affect program effectiveness, the ideal 
intervention received would closely correspond to the intended implementation. However, a 
systematic review of prevention and promotion programs found that implementation was never 
perfect, and few studies reach levels greater than 80% (Durlak & DuPre, 2008). Based on these 
results, perfect implementation is improbable for any program. Additionally, the current study 
was conducted amidst a global pandemic, causing disruption to the implementation plan. As 
such, we follow recommendations to monitor and report information on implementation in the 
current report, emphasizing the challenges faced due to COVID-19 and how these challenges 
may have impacted the results. 


Method 


Participants 

The current study involved students from Region One in South Texas and examined the 
potential relationships between SEL lessons and academic and behavioral outcomes. According 
to Region One ESC statistics, 96% of students are Hispanic, 85% are considered economically 
disadvantaged, 38% English learners, and 6% are migrants. Participants from this study came 
from a larger longitudinal sample consisting of 11,865 students from 55 middle schools where 
students complete an annual assessment measuring their SE skills. Students in this cohort were 
initially assessed when they were in seventh grade in 2019 and again when they were in eighth 
grade in 2020. During the 2020 school year, a subset of schools was invited to participate in an 
SEL implementation study. All 55 schools from the larger cohort were first categorized based on 
their 2019 SE skill levels (high-, medium-, and low-scoring schools). Schools from each level 
were then identified by Region One ESC to receive an invitation to participate. Of 18 schools 
initially selected, 14 opted to participate in the study. Across the 14 schools, a total of 1,444 
students were assigned to the intervention group to receive SEL lessons throughout the school 
year. 


Procedure 

In the spring of 2019, all participants in the study completed the Mosaic™ by ACT®: Social 
Emotional Learning assessment, which measured their baseline SE skill levels and perceptions 
of school climate. Of those who took the initial assessment, students from 14 middle schools 
were assigned SEL lessons for two of three skills: Sustaining Effort, Getting Along With Others, 
and Maintaining Composure. Figure 1 provides a depiction of the implementation plan, and 
Table 1 provides skill definitions and example lessons for each skill. Based on baseline scores 
from the 2018-2019 school year, all schools were instructed to deliver the Sustaining Effort unit 
content. Facilitators were asked to implement two mandatory Sustaining Effort lessons and then 
choose four additional lessons from a list of eight, with lessons delivered approximately once 
per week starting in the fall semester. Students from all the schools completed the lessons on 
Sustaining Effort by February or early March of 2020. For a second unit, schools were given a 
choice between Getting Along With Others or Maintaining Composure. Schools were 
encouraged to select content for this unit based on student test scores or school priorities. In 
total, the intended number of lessons for the current study was six lessons on Sustaining Effort 
and six additional lessons on one of the two remaining skills. To receive the full dosage, 
students would complete a total of 12 lessons. This target was intended to provide adequate 
programming for each target skill without overburdening facilitators. 
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Figure 1. Implementation Plan for SEL Lessons 


Teach two total units 


#1 


First Unit: Sustaining Effort (Required) Second Unit (Schools choose one) 


Sequence at a Glance 


Week 1: Introduction to Sustaining Effort Lessons 
Week 7: Second Unit Introduction Lessons 


Weeks 2-5: Choose 4 Sustaining Effort Activities 
Week 2: Sustaining Effort Activity 1 
Week 3: Sustaining Effort Activity 2 
Week 4: Sustaining Effort Activity 3 
Week 5: Sustaining Effort Activity 4 


Weeks 8-11: Choose 4 Second Unit Activities (Getting 
Along With Others or Maintaining Composure) 

Week 8: Second Unit Activity 1 

Week 9: Second Unit Activity 2 

Week 10: Second Unit Activity 3 

Week 11: Second Unit Activity 4 


h 
J [ | 


[ Sustaining Effort 


Sustaining Effort 
Introduction and 
Lecture/Discussion Material 
Introductory Video: Angela 
Duckworth’s TED Talk on 
Sustaining Effort 

What is Sustaining Effort? 
Word Wall Activity 


Sustaining Effort Activities 
Bouncing Back 
| Want to Get Better! 
That Time When You Worked 
Really, Really, Really, Really, 
Really Hard 
Did You Know? Persistence 
Stories 
Formulating a Goal 
Thinking About Goals 
The Meaning of My One-Year 
Goal 
Planning to Achieve Your 
Goals 
Circle to a Square 
Roadblocks 
Make Your Wishes Come 
True 
Using Time to Achieve Your 
Goals 


Sustaining Effort 
Culminating Activity 


Getting Along With Others 


Getting Along With Others 
Introduction and Lecture 


Getting Along With Others Group 


Activity 


Getting Along With Others 
Activities 

Individual Differences 
All Oranges Are Alike 
Differences in Perspectives 
Philosophical Chairs 
Changing Places 

Walk in My Shoes 

Magic Questions 

Body Language Speaks First 
“|” Messages 
The Trouble with Teams 
Cultural Splash 

Blindfold Square 

Mousetraps 

Positions to Common Interests 
The Belly and the Members 
Labels 

Behaviors That Leave People 
Out 

Skype Session 

Service Project 


Getting Along With 
Others 
Culminating Activity 


Maintaining Composure 


Maintaining Composure 
and Sustaining Effort: The 
Same or Different? 
Maintaining Composure 
Introduction and Lecture 
What is Maintaining 
Composure? Word Wall 
Activity 


Maintaining Composure 
Activities 

Mindfulness 
In My Zone: Locus of 
Control 
Engaging in Tough 
Conversations 
Dealing with Stress 
Name It, Aim It, Tame It, 
Solve It 
Managing Anger 
Gratitude 


Maintaining Composure 
Culminating 
Activity 
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The lesson cadence for the second unit followed that of the Sustaining Effort unit; facilitators 
were to deliver two mandatory lessons and then select four others from a list of lessons for each 
skill. Thus, strong fidelity would involve each student receiving the prescribed lessons on two 
skills in this prescribed order. Instruction was to take place over the 2019-2020 school year. All 
schools were responsible for setting their implementation schedules and creating an 
implementation plan before starting the school year. However, the implementation of the 
remaining lessons on Getting Along With Others and Maintaining Composure was impacted by 
the COVID-19 pandemic. Nevertheless, students from nine of the schools completed some 
programming on Getting Along With Others by May 2020. Students from six schools completed 
some programming on Maintaining Composure by June 2020. Lessons for both skills were 
delivered via virtual instruction. Following these lessons, some students completed the 
Mosaic™ by ACT®: Social Emotional Learning assessment again in the spring of 2020. Sample 
sizes were smaller in the final assessment than the initial assessment due to the COVID-19 
pandemic. 


In addition to SEL programming, all schools in the intervention group planned to administer 
various college and career readiness programming in line with GEAR UP (Gaining Early 
Awareness and Readiness for Undergraduate Programs), a federal grant opportunity to improve 
college preparedness in students from underrepresented groups or who would be the first to 
attend college in their families. These services were also to be implemented throughout the 
school year. These services include academic enrichment, opportunities to visit colleges and 
shadow college students, mentoring, counseling and advising, educational field trips, family 
enrichment activities, financial aid counseling, financial literacy support, job shadowing, 
leadership development, student workshops, parent workshops, and tutoring. In some cases, 
students received services based on their needs, which were determined by previous 
assessment scores on SE skills. Although these services were not part of the SEL intervention, 
this programming provides important context for interpreting the study results because it may 
have affected the study outcomes. In support of this possibility, previous research examined 
GEAR UP programming with students from Region One. The study found that although Region 
One students scored lower on measures of academic discipline, commitment, and academic 
success than a national sample when programming started in Grades 7-9, they scored 
significantly better than the national sample by Grade 12, following the implementation of GEAR 
UP programming (Albert et al., 2020). 


For both the SEL lessons and additional GEAR UP services, implementation differed by school. 
Some schools embedded SEL or GEAR UP content into core academic class time, such as 
during math or science classes. Other schools embedded content into elective subject area 
courses, such as physical education or keyboard classes. Some schools designated time during 
the school day for SEL skill-building initiatives, while others implemented the lessons as part of 
after-school programs. There was also variation in whether services were offered as universal 
services or targeted specifically to students with demonstrated areas for growth. 


Outcomes 

Students in the study completed the Mosaic™ by ACT®: Social Emotional Learning assessment 
twice. This assessment measures five SE skills: Sustaining Effort, Getting Along With Others, 
Maintaining Composure, Keeping an Open Mind, and Social Connection. In addition, the 
assessment measures two aspects of school climate: Relationships With School Personnel and 
School Safety Climate. In summary, this assessment measured several key outcomes, including 
a student's final SE skill levels and sense of school climate in the spring of 2020. Full details on 
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this assessment can be found in the technical manual (ACT, 2020). In addition to administering 
this assessment, we obtained data on grades for students who completed the SEL lessons, 
including course grades in math, English, science, and history. Further, we examined two 
behaviors disruptive to learning, absences and discipline incidents. 


Results 
Analytic Approach 


Before turning to results, we examine two key components of implementation, fidelity and 
dosage for the SEL lessons. We also examine the amount of GEAR UP programming students 
received. Next, we examine overall changes in several of the primary outcomes of interest, 
including changes in SE skills, school climate, and grades. Examining changes in these 
variables provides an overall indication of student levels at each grade. Afterward, we turn to the 
primary analyses of interest, examining the relationships between participation in SEL lessons 
and each outcome of interest while accounting for implementation differences. Specifically, our 
primary analytic approach involved hierarchical regressions predicting key outcomes for those 
assigned to the SEL lessons based on the number of hours spent on SEL lessons while 
accounting for variability in GEAR UP programming and each outcome’s initial levels. 


Implementation 
SEL Lesson Administration 


Table 3 reports the fidelity of implementation (the degree to which the program was delivered as 
intended) by presenting the extent to which the implementation plan was followed across 
schools. In most cases, implementation approached or exceeded 80%, meaning that 80% of the 
schools delivered the programming as originally intended. This level of implementation can be 
considered high fidelity given that previous research has revealed that interventions rarely meet 
this threshold. Regarding the culminating activity for the additional skill (Maintaining Composure 
or Getting Along With Others), implementation was lower at 50%, likely due to the disruption of 
the pandemic. In addition to examining the lessons provided, we also determined how closely 
the schools followed the planned order of implementation (lessons on Sustaining Effort followed 
by an additional skill). In this case, eight of the 14 schools followed this order. In summary, most 
schools followed the original implementation plan fairly closely in terms of administering lessons 
for Sustaining Effort. However, implementation of the SEL lessons for the remaining skill was 
more disrupted due to the COVID-19 pandemic. 


Table 3. Administration of SEL Lessons Based on Key Features of the Implementation Plan 
Across Schools 


Implemented at Implemented Implemented 
least one ore Tealtaretiiare) four 
Taligere[U(eave) ay activity eYeleliareyar-l| 
lesson iet-T-Yo al-3 
Number of schools for Sustaining Effort 14 11 10 
% of schools for Sustaining Effort 100% 79% 71% 
Number of schools for additional skill 12 7 10 


% of schools for additional skill 86% 50% 71% 
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SEL Lessons Received 

Despite the efforts of facilitators, students may still have chosen not to attend the lessons for 
any number of reasons. This possibility is particularly likely for lessons on Getting Along With 
Others and Maintaining Composure due to the virtual administration. Table 4 presents the 
average number of hours students spent on lessons for each skill and the average number of 
lessons received. In addition, this table presents the range of hours and lessons to demonstrate 
the variability in the dosage. For Getting Along With Others and Maintaining Composure, these 
averages and ranges reflect only the students from schools that chose to implement each skill. 
For instance, the average for Getting Along With Others is based on the students from schools 
that chose to implement Getting Along With Others lessons. In some cases, the targeted skill 
could not be determined based on the description provided. In those cases, we classified the 
activity as miscellaneous. As this table shows, students received the most programming in 
Sustaining Effort. However, the average is still below the six-lesson target given to the schools. 
Additionally, students received some instruction on Getting Along With Others and Maintaining 
Composure, but the average number of lessons fell short of the six-lesson goal for each skill. In 
total, students received an average of 8.5 lessons, which is a dosage of 71% of the target of 12 
total lessons. In sum, although the COVID-19 pandemic disrupted implementation, students 
received a relatively high dosage of lessons on Sustaining Effort (i.e., close to the intended 
number) and a somewhat lower dosage of lessons on the remaining skills. 


Table 4. Number of Lessons Received on Average for Each Skill and All Skills 


Sustaining Getting Along IM EViale-Vial ire) Miscellaneous Total Received 

Siceyas With Others Composure Lessons 
N 1,444 952 447 1,444 1,444 
Range of Hours 0-8.5 0-6 0-6 0-3 0-12.3 
Average Hours 2.8 2.2 2.2 0.2 5.3 
Range of Lessons 0-11 0-8 0-11 0-10 0-29 
Average Number of 4.4 3.6 2.8 0.1 8.5 
Lessons 


Other GEAR UP Services Received 


In addition to these services, students received various programming through GEAR UP 
initiatives. This programming included activities such as educational field trips, financial aid 
counseling, job shadowing, and student workshops. Table 5 presents the lessons received 
across various types of activities and the average time spent on each type, as well as the range 
of time and lessons to show the variability in programming received. As this table shows, 
students received a substantial amount of additional programming, with an average of almost 29 
hours spent on these activities. Of these activities, the three activities students spent the most 
time on were student workshops, counseling or advising, and college visits (average hours = 
13.0, 7.6, and 1.8, respectively). 


In addition to recording overall time, we coded these additional activities as either related or 
unrelated to SEL. To code these activities, we sent a list of the activity descriptions to each 
facilitator and asked them to indicate if the activity was related to SEL. The authors then 
reviewed these ratings and checked for consistency across campuses, making changes based 


ACT Research & ACT’s Center for Equity in Learning | August 2021 11 


on consensus when deemed necessary. Further, we also broke these activities down based on 
if they included parents. We specifically coded for parental involvement in activities because 
academic, behavioral, social, and emotional outcomes are impacted by family engagement 
(Brooks & Lambert, 2019). As such, the degree of parental involvement may have impacted 
student outcomes in the current study. We coded any activity involving a parent workshop or 
that mentioned parents in the description as involving parents. In contrast to these additional 
GEAR UP services, none of the SEL lessons examined in this study specifically involved 
parents. As Table 5 shows, students also received a substantial amount of SEL-related 
programming through GEAR UP, with an average of 7.10 hours exceeding the average time of 
5.3 hours spent on the lessons from this study. It is important to keep this time spent on SEL- 
related programming in mind when interpreting this study’s results. In particular, we only have 
records of the additional activities for students assigned to the SEL lessons. Based on the 
number of additional services received by these students, we would expect those who did not 
receive the SEL lessons to have nevertheless received a substantial amount of SEL-related 
programming through GEAR UP. 


Table 5. Dosage of GEAR UP Services Received on Average for Those Assigned to SEL 
Lessons 


Range of Average Range of Average 
hours hours lessons number of 
lessons 
Total Number Lessons 1,444 0.5-134.5 28.8 1-55 14.0 
SEL Related 1,444 0-73.8 7.1 0-14 oc 
Parent Related 1,444 0-24.8 a2 0-10 4.0 


Overall Change 
Change in SE Skills from 2019 to 2020 


Turning to our primary outcomes of interest, Table 6 presents changes in student SE skills 
broken down for schools that were and were not assigned to the SEL lessons. As this table 
shows, students decreased in several of the skills from 2019 to 2020. However, these 
decreases tended to be small. These skills included Sustaining Effort, F(1, 2848) = 4.75, p = 
.03, Np? = .002; Getting Along With Others, F(1, 2848) = 21.46, p < .01, np? = .007; Keeping an 
Open Mind, F(1, 2848) = 53.51, p < .01, np2= .018; and Social Connection, F(1, 2848) = 65.90, 
p < .01, np?= .023. For Maintaining Composure, scores remained constant from 2019 to 2020: 
F(1, 2848) = .369, p = .54, np? < .001. These patterns were similar regardless of whether 
schools were assigned to the SEL lessons, with the interaction between inclusion in the study 
and change in skills failing to reach significance in all cases (Fs < 3.7, ps > .06). Although the 
goal would be for students to increase their SE skills over time, these decreases are 
unsurprising given developmental trends for this age group, in which students often have 
declines in SE skills around Grade 8 or 9 and then experience gradual increases (Soto, John, 
Gosling, & Potter, 2011; West et al., 2020). 
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Table 6. SE Percentiles for 2019 and 2020 


Schools not in the study (n = 2,074) Schools in the study (n = 776) 


Sustaining Effort 

Getting Along With Others 
Maintaining Composure 
Keeping an Open Mind 


Social Connection 


2019 
53.69 (29.10) 
59.30 (30.56) 
53.32 (30.04) 
55.37 (28.37) 
52.15 (29.28) 


2020 
53.16 (29.53) 
56.43 (30.56) 
52.82 (29.84) 
50.06 (29.10) 
47.76 (29.75) 


2019 
52.07 (28.90) 
56.18 (29.76) 
50.93 (29.69) 
52.02 (28.01) 
49.93 (28.85) 


2020 
50.22 (29.88) 
53.81 (30.61) 
50.75 (29.75) 
48.91 (28.64) 
45.46 (29.73) 


Note: Standard deviations are presented in parentheses. 


Change in School Climate from 2019 to 2020 


In addition to grades, another key outcome was school climate. Table 7 presents changes to 
climate for schools that were and were not assigned to the SEL lessons. As this table shows, 
school climate tended to remain constant over time. For relationships, scores were stable from 
2019 to 2020: F(1, 2847) = .51, p = .47, np2< .001. Additionally, safety remained constant from 
2019 to 2020: F(1, 2848) = .07, p < .79, np2< .001. These scores were similarly constant for 
students in schools assigned to the SEL lessons and those from schools that were not assigned 
to the lessons. In neither case was the interaction between inclusion in the study and change in 
climate significant [relationships: F(1, 2847) = .10, p = .75, np? < .001; safety: F(1, 2848) = 1.70, 
p =.19, Np?= .001]. 


Table 7. School Climate Scores for 2019 and 2020 


Schools not in the study Schools in the study 


2019 2020 2019 2020 
Relationships 4.07 (0.59) 4.07 (0.62) 3.97 (0.57) 3.98 (0.60) 
Safety 4.31 (0.71) 4.32 (0.74) 4.22 (0.67) 4.20 (0.70) 


Note: Standard deviations are presented in parentheses, and scores range from 1 to 6 for each 
outcome. 


Change in Course Grades from 2019 to 2020 


Finally, we examined changes in average course grades from 2019 to 2020. Grades in 2020 
were given during the COVID-19 pandemic. Table 8 presents changes to average course 
grades, shown in percentages, in math, English, history, and science for those who were in 
schools assigned to SEL lessons. We did not obtain grade data for students from schools that 
did not participate in this study. As this table shows, grades were fairly consistent from 2019 to 
2020. Small decreases did reach significance for three subjects. For English, grades decreased 
on average from 2019 to 2020: t(1179) = -4.29, p < .01, d = -.13. Additionally, history grades 
decreased on average from 2019 to 2020: t(1199) = -5.32, p < .01, d= -.15. Further, science 
grades decreased on average from 2019 to 2020: t(1207) = -3.71, p < .01, d= -.11. In contrast, 
math grades remained constant from 2019 to 2020: t(955) = -1.34, p = .18, d= -.04. 
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Table 8. Course Grades for 2019 and 2020 for Schools in the Study 


7AUh be) YIN) 


Math 79.90 (9.08) 80.24 (9.00) 
English 83.13 (8.98) 82.19 (8.88) 
History 85.29 (7.74) 84.27 (8.79) 
Science 84.60 (8.75) 83.88 (9.37) 


Note: Standard deviations are presented in parentheses, and values shown are percentages out 
of a possible 100. 


Effects of Participation in SEL Lessons 

Predictors of 2020 SE Skill Scores 

The next set of analyses focuses only on students in the intervention group who received 
additional SEL programming during the 2019-2020 school year. We were interested in 
determining how 2019 SE skill scores, time spent on SEL lessons, and time spent on GEAR UP 
programming relate to 2020 SE skill percentile scores. Table 9 reports correlations between SE 
skills in 2020 and SE skills in 2019, total hours of additional GEAR UP activities in 2020, and 
time spent on lessons for each skill. As this table shows, 2019 percentile scores for each SE 
skill were most strongly related to the 2020 percentile scores. In addition, GEAR UP hours were 
related to SE percentile scores in 2020. In contrast, hours spent on SEL lessons did not 
consistently relate to SE percentiles in 2020. In one case, time spent on Sustaining Effort 
lessons in 2020 was negatively correlated with Sustaining Effort percentiles in 2020. Given the 
relationship between Sustaining Effort percentiles each year, one potential explanation for this 
finding is that students who had lower initial scores on Sustaining Effort were provided with 
additional practice on this skill. In support of this possibility, Sustaining Effort percentiles in 2019 
were also negatively related to time spent on Sustaining Effort lessons in 2020 (r= -.09, 

p = .03). 
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Table 9. Correlations Between Each SE Skill, GEAR UP Hours, and Time Spent on Relevant 
Lessons 


N Skill percentile GEAR UP a leit:e) ames) = 


in 2019 hours in (eX--Yo) al  co) g 
yA) rT ey el iCer-le)(--J ¢1| 

Sustaining Effort percentile in 2020 386 .58* .16* = 16" 
Getting Along With Others percentile 386 .58* 12" 0.06 

in 2020 

Maintaining Composure percentile in 386 .59* .12* 0.01 

2020 

Note: *p < .05 


To determine if hours spent on SEL lessons predict final SE percentile scores, we fit three 
hierarchical regression models to predict the final level of each of the target skills. In each case, 
the percentile score on each skill in the spring of 2020 served as the outcome, and hours spent 
on SEL lessons targeting that skill was the predictor. These analyses were restricted to students 
from the 14 schools assigned to implement the SEL lessons. In addition, each model controlled 
for students’ initial SE skill levels in 2019 and hours spent on GEAR UP activities. The key 
results for these regression models are presented in Table 10. As these models show, the 
strongest predictors of SE skill percentile scores in 2020 were students’ initial percentile scores 
in 2019. In contrast, the other variables generally did not predict percentile scores in 2020. 


For Sustaining Effort, percentile scores in 2019 accounted for a significant amount of variance in 
percentile scores in 2020 (R2 = .33, F[1, 384] = 191.64, p < .01). Adding time spent on GEAR 
UP activities into the model accounted for a small amount of additional variance (R2 = .34, AR? = 
.007, AF = 4.11, p = .04). However, adding time spent on Sustaining Effort lessons into the 
model did not account for a significant amount of additional variance (R2 = .35, AR? = .007, AF = 
3.83, p = .05) despite marginal significance. For Getting Along With Others, percentile scores in 
2019 accounted for a significant amount of variance in percentile scores in 2020 (R2 = .34, F[1, 
384] = 197.08, p < .01). Adding time spent on GEAR UP activities into the model did not account 
for a significant amount of additional variance (R2 = .34, AR? = .004, AF = 2.61, p= .11). 
Similarly, adding time spent on Getting Along With Others lessons into the model did not 
account for a significant amount of additional variance (R2 = .34, AR2 = .003, AF = 1.80, p = .18). 
For Maintaining Composure, percentile scores in 2019 accounted for a significant amount of 
variance in percentile scores in 2020 (R2 = .34, F[1, 384] = 200.55, p < .01). Adding time spent 
on GEAR UP activities into the model did not account for a significant amount of additional 
variance (R2 = .35, AR? = .003, AF = 2.03, p = .16). Similarly, adding time spent on Maintaining 
Composure lessons into the model did not account for a significant amount of additional 
variance (R2 = .35, AR? = .003, AF = 1.98, p = .16). 
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Table 10. Predicting Final SE Percentile Scores for Target Skills Based on Hours on Lessons, 
Initial Levels, and GEAR UP Hours 


WM Coxe (=) Predictor B t Tre faliitere alec 
I 


Sustaining Effort percentile score in 2020 


1 Sustaining Effort percentile 2019 08 13.84 <.001 
= a eenees Sea ae meme er erates | = ene ae _ 
2 GEAR UP hours in 2020 .08 2.03 .04 
a : SS CS 
3 GEAR UP hours in 2020 .05 1.92 23 
3 Time on Sustaining Effort lessons -.09 =1.96 05 


Getting Along With Others percentile score in 2020 


1 Getting Along With Others percentile 08 14.04 <.001 
2019 

2 Getting Along With Others percentile 08 13.85 <.001 
2019 

2 GEAR UP hours in 2020 .07 1.62 ma 

3 Getting Along With Others percentile 08 13.92 <.001 
2019 

3 GEAR UP hours in 2020 04 .90 OT 

3 Time on Getting Along With Others .06 1.34 18 

lessons 


Maintaining Composure percentile score in 2020 


1 Maintaining Composure percentile .59 14.16 <.001 
2019 

2 Maintaining Composure percentile 58 13.97 <.001 
2019 

2 GEAR UP hours in 2020 .06 1.43 16 

3 Maintaining Composure percentile 58 14.02 <.001 
2019 

3 GEAR UP hours in 2020 .08 1.76 .08 

3 Time on Maintaining Composure .06 1.41 16 


lessons 
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Predictors of 2020 School Climate Scores 


Table 11 reports correlations between the two dimensions of school climate (Relationships With 
School Personnel and School Safety) in 2020 and school climate in 2019, total hours of 
additional GEAR UP activities in 2020, and total hours spent on SEL lessons. As with the SE 
skills, initial scores in 2019 were most strongly related to both dimensions of school climate in 
2020. Additionally, GEAR UP hours in 2020 were positively related to School Safety in 2020. In 
contrast, time spent on SEL lessons was unrelated to final levels of either dimension of school 
climate. 


Table 11. Correlations Between School Climate, Hours on Lessons, Initial Scores, and GEAR 
UP Hours 


eYor- | (sMoxexe) c= a] GEAR UP hours Total time on SEL 
2019 TawAlyAl) lessons 
Relationships with 385 .40* .03 02 
School Personnel in 
2020 
School Safety in 2020 386 .41* .14* .00 
Note: *p < .05 


To determine if these variables predict the two aspects of school climate in 2020, we fit two 
hierarchical regression models similar to those used to predict percentile scores. Specifically, 
we predicted each school climate component in 2020 (i.e., Relationships With School Personnel 
and School Safety) from the total number of SEL lesson hours students received. In addition, 
initial levels of school climate in 2019 and time spent on GEAR UP activities in 2020 were 
added to each model to account for these variables. Results from these models are presented 
in Table 12. As these models show, initial levels of school climate dimensions in 2019 most 
strongly predict levels of both components in 2020. In addition, hours spent on GEAR UP 
activities predicted School Safety in 2020 when accounting for School Safety in 2019. 


For Relationships With School Personnel, scale scores in 2019 accounted for a significant 
amount of variance in scale scores in 2020 (R2 = .16, F[1, 383] = 72.04, p < .01). Adding time 
spent on GEAR UP activities into the model did not account for a significant amount of 
additional variance (R2 = .16, AR? < .001, AF = 0.03, p = .87). Similarly, adding time spent on 
SEL lessons did not account for a significant amount of additional variance (R2 = .16, AR2 = 
.002, AF = .88, p = .35). For School Safety, scale scores in 2019 accounted for a significant 
amount of variance in scale scores in 2020 (R2 = .17, F[1, 384] = 78.07, p < .01). Adding time 
spent on GEAR UP activities into the model accounted for a small amount of additional variance 
(R2 = .19, AR2 = .019, AF = 9.17, p < .01). However, adding time spent on SEL lessons did not 
account for a significant amount of additional variance (R2 = .19, AR2= .003, AF = 1.23, p = .27). 
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Table 12. Predicting Final School Climate Based on Hours on Lessons, Initial Levels, and 
GEAR UP Hours 


IM Coxe [1] Predictor B t STefaniiler- Later 


Relationships With School Personnel scale scores in 2020 


1 Relationships 2019 40 8.49 <.001 
E 2  ~—~—-Relationships2019 si (asti(ié‘é‘ awowé~}])™lU845 KOT 
2 GEAR UP hours in 2020 .01 0.17 87 
ae ee ee ‘ as aes 
3 GEAR UP hours in 2020 02 0.34 14 
3 Time on SEL lessons 04 0.94 Rote) 


School Safety scale scores in 2020 


1 Safety 2019 41 8.84 <.001 

2 ———————— oC Ee ee 
2 GEAR UP hours in 2020 14 3.03 <.01 

= ee 2. 2 ee eee eo : = os 
3 GEAR UP hours in 2020 Ad5 3.18 <.01 
3 Time on SEL lessons 05 11 27 


Predictors of 2020 Course Grades 


Table 13 reports correlations between course grades in 2020 and course grades in 2019, total 
hours of additional GEAR UP activities in 2020, and total hours spent on SEL lessons. As this 
table shows, grades in 2019 are most strongly related to grades in 2020. In addition, GEAR UP 
hours in 2020 are significantly correlated with course grades for three of the four subjects. 
Further, time spent on SEL lessons is significantly correlated with grades for three of the four 
subjects. 
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Table 13. Correlations Between Course Grades in 2020, Hours on SEL Lessons, Course 
Grades in 2019, and GEAR UP Hours 


Grades in 2019 GEAR UP hours Total time on SEL 


in 2020 lessons 
Math 954 .63* A7* 05 
English 1,178 .65* .09* .06* 
History 1,198 .69* .04 .07* 
Science 1,206 .72* 15* .06* 
Note: *p < .05 


To determine if these variables predicted course grades in 2020, we fit four hierarchical 
regression models. These models were similar to those previously described, predicting grades 
in 2020 for each core class from the total number of hours spent on SEL lessons. In addition, 
grades in 2019 and time spent on GEAR UP activities in 2020 were added to each model to 
account for these variables. Table 14 presents the results of these analyses. As this table 
shows, grades in 2019 were the strongest predictors of grades in 2020. For math, time spent on 
SEL lessons remained a significant, positive predictor when accounting for initial grades and 
GEAR UP activities. Additionally, GEAR UP hours remained a significant, positive predictor of 
math and English grades when accounting for initial grades and SEL hours. 


For math, grades in 2019 accounted for a significant amount of variance in student grades in 
2020 (R2 = .40, F[1, 952] = 627.97, p < .01). Adding time spent on GEAR UP activities into the 
model accounted for a small amount of additional variance (R2 = .40, AR? = .003, AF = 5.28, p = 
.02). Additionally, adding time spent on SEL lessons into the model accounted for a small 
amount of additional variance (R2 = .41, AR? = .008, AF = 13.28, p < .01). For English, student 
grades in 2019 accounted for a significant amount of variance in student grades in 2020 (R2 = 
2, F[1, 1176] = 850.35, p < .01). Adding time spent on GEAR UP activities into the model 
accounted for a small amount of additional variance (R2 = .43, AR2= .005, AF = 11.01, p = .01). 
However, adding time spent on SEL lessons into the model did not account for a significant 
amount of additional variance (R2 = .43, AR2< .001, AF = 0.10, p = .75). For history, student 
grades in 2019 accounted for a significant amount of variance in student grades in 2020 (R2 = 
47, F[1, 1196] = 1057.43, p < .01). Adding time spent on GEAR UP activities into the model did 
not account for a significant amount of additional variance (R2 = .47, AR? = .001, AF = 3.23, p = 
.08). Similarly, adding time spent on SEL lessons into the model did not account for a significant 
amount of additional variance (R2 = .47, AR2= .002, AF = 3.40, p = .07). For science, student 
grades in 2019 accounted for a significant amount of variance in student grades in 2020 (R2 = 
.52, F[1, 1204] = 1309.30, p < .01). Adding time spent on GEAR UP activities into the model did 
not account for a significant amount of additional variance (R2 = .52, AR? < .001, AF = 0.03, p = 
.87). Similarly, adding time spent on SEL lessons did not account for a significant amount of 
additional variance (R2 = .52, AR? < .001, AF = 0.98, p = .32). 
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Table 14. Predicting 2020 Grades Based on SEL Hours on Lessons, Initial Grades, and GEAR 
UP Hours 


NM Coxe [1] Predictor B t Tre Taiter- laters 
Math grades in 2020 


1 Math grades in 2019 63 25.06 <.001 

4 2  — Mathgradesin2019— i sst—“‘<«<CS KH 
2 GEAR UP hours in 2020 .06 2.30 02 

: ee ee on ie ede ee see ee ee mie eats 
3 GEAR UP hours in 2020 .08 3.00 <.01 
3 Time on SEL lessons .09 3.64 <.001 


English grades in 2020 


1 English grades in 2019 65 29.16 <.001 
7 ———=— SC 
2 GEAR UP hours in 2020 .07 3.32 .001 
a en a =r —— eee — 
3 GEAR UP hours in 2020 .07 3.31 .001 
3 Time on SEL lessons 01 0.32 15 


History grades in 2020 


1 History grades in 2019 .69 32.52 <.001 
- a - -aeen we ae ae 
2 GEAR UP hours in 2020 -.04 -1.8 .07 
SS ee ne ee 
3 GEAR UP hours in 2020 -.03 -1.36 17 
3 Time on SEL lessons 04 1.84 .07 


Science grades in 2020 


1 Science grades in 2019 a4 36.18 <.001 

7 ee 
Z GEAR UP hours in 2020 .003 0.17 87 

4 3. ~—s Science grades in2019. i (itsti‘é~Sx]:*~*~*«*« SCO KO 
3 GEAR UP hours in 2020 01 0.39 70 
3 Time on SEL lessons 02 1.00 2 
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Predictors of 2020 Absences and Discipline Incidents 

Table 15 reports correlations between absences in 2020 and absences in 2019, total hours of 
additional GEAR UP activities in 2020, and total hours spent on SEL lessons. As this table 
shows, absences in 2020 were most strongly related to absences in 2019. In addition, GEAR 
UP hours in 2020 were related to fewer absences in 2020. However, time spent on SEL lessons 
was not significantly related to absences in 2020. 


Table 15. Correlations Between Absences in 2020, Absences in 2019, Time on SEL Lessons, 
and GEAR UP Hours 


N Absences in 2019 GEAR UP hours in Total time on SEL 
LI40) lessons 


Absences in 2020 1,282 .64* =20* 04 


Note: *p < .05 


To determine if these variables predicted absences in 2020, we fit a hierarchical regression 
model. This model predicted absences in 2020 from the total number of hours spent on SEL 
lessons. In addition, absences in 2019 and time spent on GEAR UP activities in 2020 were 
added to each model to account for these variables. Table 16 presents the results of these 
analyses. As this table shows, absences in 2019 were the strongest predictors of absences in 
2020. Student absences in 2019 accounted for a significant amount of variance in student 
absences in 2020 (R? = .40, F[1, 1280] = 862.63, p < .01). Adding time spent on GEAR UP 
activities into the model accounted for a significant amount of additional variance (R2 = .44, AR? 
= .034, AF = 77.99, p < .01). However, adding time spent on SEL lessons into the model did not 
account for a significant amount of additional variance despite the trend in the expected 
direction (R2 = .44, AR2= .001, AF = 3.01, p = .08). 


Table 16. Predicting 2020 Absences Based on SEL Hours on Lessons, Absences in 2019, and 
GEAR UP Hours 

iM Coxe (=1| Predictor B t STefaiiler- laters. 
Absences in 2020 


1 Absences 2019 64 29.37 <.001 
2 Absences 2019 i t—(<‘i‘z SS 3005 <.001 
2 GEAR UP hours in 2020 -.19 -8.83 <.001 
es 3 Se SS SSS 
3 GEAR UP hours in 2020 -.20 -8.97 <.001 
3 Time on SEL lessons -.04 -1.74 .08 


Further, based on previously established guidelines, we broke the participants from schools who 
implemented the SEL lessons into three groups (Attendance Works, n.d.). The groups included 
those with acceptable absentee records (fewer than 10 days missed), habitual absentee records 
(at least 10 days but fewer than 18), and chronic absentee records (more than 18 days missed). 
Table 17 reports these results. As this table shows, most students fell into the acceptable range 
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of absences in 2020, with very few students in the chronic range. To examine potential 
differences in SEL lesson hours based on absentee group, we ran a one-way ANOVA with SEL 
lesson hours as the outcome and absentee group as the independent variable. As reported in 
Table 17, no differences occurred based on absentee group. 


Table 17. Differences in SEL Lesson Hours Based on Absenteeism 


Acceptable Habitual Chronic 
M SD N M SD N M SD N F 
SEL lesson 5.54 3.39 15222 5.13 2.84 61 5.33 3.68 8 429 


hours 


Finally, we examined district discipline incidents. Given relatively low variability in discipline 
incidents, we dichotomized this variable as either zero incidents or one or more incidents. We fit 
a binary logistic regression model to predict discipline incidents in 2020 based on hours spent 
on SEL lessons. We also controlled for discipline incidents in 2019 and hours spent on GEAR 
UP lessons in 2020. Having at least one discipline incident in 2020 was coded as one, and not 
having an incident was coded as zero. The overall model was significant: x? (3) = 211.02, p = 
.01. As Table 18 shows, the odds of a student who had a discipline incident in 2019 having a 
discipline incident in 2020 were 11.82 times higher than for a student who did not have a 
discipline incident in 2019. Additionally, the odds of a student who had more hours of SEL 
lessons in 2020 having a discipline incident in 2020 were lower than for a student who did not 
have as many hours of SEL lessons. In contrast, the odds of a discipline incident in 2020 were 
not significantly predicted by GEAR UP hours in 2020. In sum, as expected, students who had a 
discipline incident in 2019 were more likely to have a discipline incident in 2020, but those who 
completed more hours of SEL lessons were less likely to have a discipline incident in 2020. 


Table 18. Predicting 2020 Discipline Incidents Based on SEL Hours on Lessons, Initial Levels, 
and GEAR UP Hours 
iM Coke (=)) Predictor B Wald Significance 40) (53) 


Discipline Incidents 2020 


1 Discipline incidents 2019 2.44 194.80 <.001 11.41 
a oo ee — os ers oe ae 
2 GEAR UP hours in 2020 -0.004 1.81 18 1.00 
eee : ee ee 
3 GEAR UP hours in 2020 -0.01 3.31 .07 0.99 
3 Time on SEL lessons -0.06 5.43 .02 0.94 
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Discussion 


The current study examined how the implementation of SEL lessons affected a group of 
primarily Latinx students while accounting for other services these students were receiving. 
Initially, 14 schools were selected to implement SEL lessons throughout the 2019-2020 school 
year. In the spring of 2020, COVID-19 caused schools to close, and a shift to online learning 
was required. Despite this disruption, dedicated facilitators worked to implement the SEL 
lessons online and continued the study. With this effort, the facilitators delivered much of the 
initially intended programming, resulting in fairly high fidelity. 


Although the facilitators provided many lessons, the actual dosage of lessons was relatively low. 
Overall, students received an average of 8.5 lessons, or a dosage of 71% of the 12-lesson 
target. Variability in the dosage was also high due to some students attending far fewer lessons 
and others attending far more. This low dosage may have reduced the effects of the SEL 
lessons. Still, even under typical conditions, most studies report suboptimal implementation 
(Durlak & DuPre, 2008). As such, the dosage of lessons received in the current study seems 
reasonable, especially considering the difficulty monitoring and enforcing attendance in an 
online environment. 


Nevertheless, the overall number of the SEL lessons was relatively small when compared to the 
amount of other programming students in this study received. In particular, many activities were 
offered as part of GEAR UP, with students attending an average of close to 29 hours of 
activities, including around seven hours of SEL-related activities. In comparison, students 
received an average of around five hours on the SEL lessons. Additionally, students all received 
a substantial amount of instruction on academic subjects and electives. Although we did not 
obtain records of these hours, a survey conducted by the U.S. Department of Education found 
that in 2007—2008, students spent an average of 21.9 hours on instruction for core academic 
subjects and 31.2 hours on instruction in total each week (U.S. Department of Education, 2008). 
Additionally, the minimum amount of instructional time per school year in Texas was 1,260 hours 
in 2018 (Education Commission of the States, 2020). Further, in the current study, some schools 
prioritized academic instruction due to the disruption of COVID-19. Within the context of all this 
programming and instruction, five hours spent on SEL lessons spread throughout the year is a 
relatively small amount. Nevertheless, several results of the current study are promising. 


One promising result of the current study was the relative consistency in key outcomes from 
2019 to 2020. Despite the shift to online learning, school climate scores in terms of safety and 
relationships remained constant across this time. Additionally, scores for Maintaining 
Composure remained constant across this time, while scores on the remaining SE skills showed 
slight declines. However, larger and more consistent declines in skills may have been expected 
based on developmental trends showing declines in SE skills within this age range (West et al., 
2020). Additionally, grades in three courses declined, but these effects were small, with the 
largest decline being around 1% in history. Here also, larger declines may have been expected 
based on recent research by Curriculum Associates (2021) showing that fewer students are 
ready for grade-level work during COVID-19 than were in the three previous years. Curriculum 
Associates (2021) also found that COVID-19 most impacted students from underserved 
communities and that the impact was larger for math than reading. The current study, in 
contrast, involved a group of primarily Latinx learners who might also be described as 
underserved and found math grades remained constant from 2019 to 2020. Although this 
consistency is promising, the current study cannot rule out the possibility that more leniency was 
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given while grading in 2020. Still, even when increased leniency is considered, it does not 
explain consistency in the other outcomes. 


Another promising result from the current study was the benefits associated with GEAR UP 
programming. Hours spent on GEAR UP activities in 2020 significantly predicted higher grades 
in math and English in 2020. Additionally, hours spent on GEAR UP activities significantly 
predicted greater Sustaining Effort percentiles in 2020 and School Safety in 2020. Finally, GEAR 
UP activities significantly predicted fewer absences in 2020. These relationships were 
demonstrated in all of these cases while controlling for initial levels of these outcomes in 2019. 
These results are consistent with previous research showing the benefits of GEAR UP 
programming (Albert et al., 2020). 


Although somewhat mixed, several promising results were also found for the SEL lessons. 
Following the SEL lessons, students reacted positively, saw value in these lessons, and 
reported greater content knowledge and the ability to apply lesson content in and outside the 
classroom (Brief 2). Yet relationships between these lessons and the more long-term outcomes 
reported in the current document are mixed. Hours spent on SEL lessons did not significantly 
predict several outcomes, including final levels of SE skills and school climate. Still, in most 
cases, the results of the regression models were trending in the expected direction. Further, in 
two cases (math grades in 2020 and odds of having a discipline incident in 2020), hours spent 
on SEL lessons helped predict target outcomes. Replicating these results under a more 
traditional instructional setting is an important next step for research. Nevertheless, these initial 
results are promising given the low dosage of lessons and the disruption faced due to 
COVID-19. 


Additionally, the full impact of these lessons may become more apparent when outcomes are 
examined after a longer delay. According to Yeager & Walton (2011), understanding the long- 
term influence of an intervention requires an understanding of how that intervention influences 
factors already in place at school, such as peer relationships. For instance, the SEL lessons on 
Getting Along With Others in the current study may have influenced students’ relationships with 
their teachers or peers. These influences, in turn, could lead to greater benefits over time. 
Indirect support for this possibility comes from previous research with students from Region One 
(Albert et al., 2020). In this study, GEAR UP programming took time to produce change, with 
larger benefits shown over time. As such, directly examining the long-term effects of the SEL 
lessons from the current study is an important direction for future research, especially because 
the outcomes in the current study were examined shortly after students were given the lessons 
(e.g., Maintaining Composure lessons were not finished until June for some students). Related 
to this point, certain outcomes in the current study, such as school climate and student SE skills, 
may be expected to change more gradually. In contrast, outcomes related to more immediate 
behavior, such as discipline incidence, may be expected to respond more quickly to the SEL 
lessons. As such, the relationship between SEL lessons and discipline incidents in 2020 may be 
considered a particularly promising finding. 


Taken together, these results and the results of surveys conducted for this study underscore 
several important implications for SEL programming. First, parents and educators generally 
have positive views about SEL programming (Brief 1 and Brief 5). Additionally, students in the 
current study reacted positively to the SEL lessons (Brief 2). In some cases, time spent on the 
SEL lessons predicted more distal outcomes despite the low dosage of programming. These 
promising results show that all relevant stakeholders see value in SEL programming, but time 
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and effort are required to provide the sustained implementation of programming that may be 
necessary to produce long-term benefits to students. Increasing the amount of programming is 
an important next step for future research. A greater emphasis on SEL programming is also 
consistent with a whole-child approach to education that goes beyond instruction on core 
academic content areas. Findings from the current study also underscore the importance of a 
structured implementation plan and continued monitoring of the fidelity and dosage of 
implementation. 


Conclusions 


This research project, in collaboration with Region One ESC, provides the following main 
conclusions: 


* Parents, students, and educators see value in SEL programming but need resources to 
implement structured SEL programming (Briefs 1, Brief 2 and Brief 5). 


* Inthe current study, the SEL activities were designed to follow SAFE recommendations, 
and the fidelity of implementation was relatively high, while the dosage was somewhat 
lower, and there was a lot of variability in this dosage. 


* Despite the low dosage, hours spent on the SEL lessons helped predict two of the target 
outcomes: improved math grades and decreased odds of a disciplinary incident in 2020. 


* Replicating these results with a more traditional instructional approach and increasing 
the amount of SEL programming are important next steps for future research. 


* Students in this study received a greater amount of programming through GEAR UP 
activities, and hours spent on these activities predicted several key outcomes, including 
improved grades in math and English, increased Sustaining Effort percentiles, increased 
school safety, and a lower rate of absences in 2020. 


* Agreater amount of SEL programming may also be necessary for associations between 
this programming and certain student outcomes like school climate. 
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